Can Fault Prediction Models and Metrics be Used for Vulnerability Prediction?

Finding security vulnerabilities requires a different mindset than finding general faults in software thinking like an attacker. Therefore, security engineers looking to prioritize security inspection and testing efforts may be better served by a prediction model that indicates security vulnerabilities rather than faults. At the same time, faults and vulnerabilities have commonalities that may allow development teams to use traditional fault prediction models and metrics for vulnerability prediction. The goal of our study is to determine whether fault prediction models can be used for vulnerability prediction or if specialized vulnerability prediction models should be developed when both are built with traditional metrics of complexity, code churn, and fault history. We have performed an empirical study on a widely-used, large open source project, the Mozilla Firefox web browser, where 20% of the source code files have faults and only 3% of the files have vulnerabilities. Both the fault prediction model and the vulnerability prediction model predicted vulnerabilities with high recall (over 90%) and low precision (9%). The precision from these vulnerability predictions was much lower than the precision from fault prediction (47%). Our results suggest that fault prediction models based upon traditional metrics can be substituted for specialized vulnerability prediction models, but requires significant improvement to reduce false positives. KeywordsSoftware metrics; complexity metrics; fault prediction; vulnerability prediction; open source projects